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The  NATO  Science  and  Technology  Organization 


Science  &  Technology  (S&T)  in  the  NATO  context  is  defined  as  the  selective  and  rigorous  generation  and  application  of 
state-of-the-art,  validated  knowledge  for  defence  and  security  purposes.  S&T  activities  embrace  scientific  research, 
technology  development,  transition,  application  and  field-testing,  experimentation  and  a  range  of  related  scientific 
activities  that  include  systems  engineering,  operational  research  and  analysis,  synthesis,  integration  and  validation  of 
knowledge  derived  through  the  scientific  method. 

In  NATO,  S&T  is  addressed  using  different  business  models,  namely  a  collaborative  business  model  where  NATO 
provides  a  forum  where  NATO  Nations  and  partner  Nations  elect  to  use  their  national  resources  to  define,  conduct  and 
promote  cooperative  research  and  information  exchange,  and  secondly  an  in-house  delivery  business  model  where  S&T 
activities  are  conducted  in  a  NATO  dedicated  executive  body,  having  its  own  personnel,  capabilities  and  infrastructure. 

The  mission  of  the  NATO  Science  &  Technology  Organization  (STO)  is  to  help  position  the  Nations’  and  NATO’s  S&T 
investments  as  a  strategic  enabler  of  the  knowledge  and  technology  advantage  for  the  defence  and  security  posture  of 
NATO  Nations  and  partner  Nations,  by  conducting  and  promoting  S&T  activities  that  augment  and  leverage  the 
capabilities  and  programmes  of  the  Alliance,  of  the  NATO  Nations  and  the  partner  Nations,  in  support  of  NATO’s 
objectives,  and  contributing  to  NATO’s  ability  to  enable  and  influence  security  and  defence  related  capability 
development  and  threat  mitigation  in  NATO  Nations  and  partner  Nations,  in  accordance  with  NATO  policies. 


The  total  spectrum  of  this  collaborative  effort  is  addressed  by  six  Technical  Panels  who  manage  a  wide  range  of 
scientific  research  activities,  a  Group  specialising  in  modelling  and  simulation,  plus  a  Committee  dedicated  to 
supporting  the  information  management  needs  of  the  organization. 
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These  Panels  and  Group  are  the  power-house  of  the  collaborative  model  and  are  made  up  of  national  representatives  as 
well  as  recognised  world-class  scientists,  engineers  and  information  specialists.  In  addition  to  providing  critical 
technical  oversight,  they  also  provide  a  communication  link  to  military  users  and  other  NATO  bodies. 


The  scientific  and  technological  work  is  carried  out  by  Technical  Teams,  created  under  one  or  more  of  these  eight 
bodies,  for  specific  research  activities  which  have  a  defined  duration.  These  research  activities  can  take  a  variety  of 
forms,  including  Task  Groups,  Workshops,  Symposia,  Specialists’  Meetings,  Lecture  Series  and  Technical  Courses. 
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Disposable  Multi-Sensor  Unattended  Ground  Sensor 
Systems  for  Detecting  Personnel 
(STO-TR-SET-158) 

Executive  Summary 

The  scope  of  the  effort  conducted  by  the  Task  Group  (SET-158  RTG)  on  Disposable  Multi-Sensor 
Unattended  Ground  Sensor  Systems  for  Detecting  Personnel  identifies  and  accesses  potential  technologies 
for  detection  of  people  that  improves  security  for  wide  area  situational  awareness  in  open  terrain  operations. 
While  the  results  are  applicable  to  the  urban  environment,  the  effort  focused  on  remote  desert  environments 
that  included  well-known  false  targets,  primarily  animals.  The  effort  took  advantage  of  existing  research 
programs  on  sensor  systems  and  as  well  as  novel  sub-systems  components  for  personnel  detection. 
The  primary  sub-system  components  include  sensing  elements,  signal  processing  and  fusion  algorithms. 
Phenomenology  aspects  were  the  primary  driver  in  each  of  these  components. 

This  work  addresses  the  ever-increasing  demand  for  remote,  robust  and  high-performance  sensor  systems  for 
surveillance  and  target  acquisition  on  the  battlefield.  The  Task  Group  placed  emphasis  on  low-cost 
unattended  sensor  technologies  to  support  the  need  for  large  quantities  of  inexpensive  multi-sensor  UGS  to 
detect  personnel.  The  focus  of  the  research  is  to  develop  robust  signal  processing  and  fusion  algorithms  using 
low-cost  multi-sensor,  non-imaging  and  imaging  sensor  nodes  distributed  along  a  remote  border  to  detect 
personnel  and  animals. 

The  goal  and  objectives  set  forth  were  carried  out  in  two  phases.  First  was  to  identify  sensors  that  align  with 
the  ideas  of  low  cost,  robust  and  data  output  capable  of  personnel  and  animal  detection  and  characterization. 
This  was  accomplished  using  national  surveys  and  a  review  of  personnel  detection  sensors  dating  back  to 
the  conflict  in  Southeast  Asia  between  the  years  of  1956  -  1975.  The  sensors  included  in  the  group’s  effort 
include  audio  and  ultrasonic  microphones,  geophones,  passive  infrared,  magnetometers,  radar  and  sonar, 
pyroelectric  array  profilers  and  both  low-  and  high-resolution  cameras.  In  the  second  phase,  sensor 
phenomenology  and  signal  processing  approaches  were  reviewed  and  developed  at  the  individual  sensor 
level.  An  international  data  collection  field  exercise  held  in  March  of  2012  on  the  SW  US  border  was  well 
attended  by  participating  Nations.  This  included  SASNet  from  Canada  and  Pearls  of  Wisdom  system  from 
Israel  operated  in  conjunction  with  ARE  personnel.  The  primary  objectives  of  the  trial  were  to  provide 
opportunities  for  the  collection  of  phenomenological  data  and  to  make  data  available  for  algorithm  testing. 

SET-158  Task  Group  met  on  numerous  occasions  beginning  with  the  initial  kick-off  meeting  held  at  the 
RTO  office  in  Paris  Spring  of  2010.  Two  additional  technical  committee  meetings  were  held  after  the  PBM 
meetings  in  Greece,  October  2010  and  Quebec  City,  May  2012.  SET-158  co-sponsored  the  4th  technical 
meeting  with  SET-TG-153  and  SET-TG-142  in  Cardiff,  Wales,  on  11  October  2011.  An  additional  three 
committee  meetings  were  held  in  conjunction  with  data  collection  field  trials  and  workshops. 

The  committee  organized  two  personnel  detection  workshops  that  were  held  in  the  USA.  Both  of  these  were 
in  conjunction  with  the  US  Army  Research  Laboratory.  These  activities  were  well  attended  with  numerous 
papers  on  personnel  detection  being  presented.  More  than  two  dozen  peer-reviewed  papers  have  been 
published  as  a  result  of  the  committee  activities. 
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Systemes  de  detection  multi-capteurs  terrestres 
autonome  destines  a  detecter  du  personnel 

(STO-TR-SET-158) 

Synthese 

Les  travaux  menes  par  le  groupe  de  travail  «  Systemes  de  detection  multi-capteurs  terrestres  autonome 
destines  a  detecter  du  personnel  »  (SET-158  RTG)  identifient  et  accedent  aux  technologies  potentielles  de 
detection  des  personnes,  qui  ameliorent  la  securite  par  1’ appreciation  de  situation  de  zones  etendues  dans  les 
operations  en  terrain  decouvert.  Meme  si  les  resultats  peuvent  s’appliquer  a  l’environnement  urbain, 
les  recherches  se  sont  concentrees  sur  les  environnements  desertiques  recules  comportant  de  faux  objectifs 
bien  connus,  principalement  des  animaux.  Les  travaux  ont  tire  parti  des  programmes  de  recherche  existants 
sur  les  systemes  de  detection  et  sur  les  tout  nouveaux  composants  de  sous-systemes  de  detection  de 
personnel.  Les  principaux  composants  des  sous-systemes  incluent  les  elements  de  detection,  le  traitement  du 
signal  et  les  algorithmes  de  fusion.  Les  aspects  de  phenomenologie  ont  ete  le  principal  moteur  de  chacun  de 
ces  composants. 

Ce  document  repond  a  la  demande  sans  cesse  croissante  de  systemes  de  detection  a  distance,  qui  soient 
robustes  et  de  haute  performance  afm  d’assurer  la  surveillance  et  l’acquisition  des  objectifs  sur  le  champ  de 
bataille.  Le  groupe  de  travail  a  insiste  sur  les  technologies  de  detection  autonomes  a  faible  cout  afm  de 
repondre  au  besoin  de  grandes  quantites  de  capteurs  terrestres  autonomes  (UGS)  peu  onereux  a  multiples 
capteurs  servant  a  la  detection  de  personnel.  L’objet  des  recherches  est  d’elaborer  des  algorithmes  robustes 
de  fusion  et  de  traitement  du  signal  utilisant  des  nceuds  de  multiples  capteurs  imageurs  et  non  imageurs  peu 
couteux  repartis  sur  une  frontiere  eloignee  pour  detecter  des  personnes  et  des  animaux. 

Les  objectifs  enonces  ont  ete  realises  en  deux  phases.  La  premiere  a  consiste  a  identifier  les  capteurs  qui 
correspondent  au  faible  cout,  a  la  robustesse  et  a  la  production  de  donnees  pennettant  de  detecter  et 
caracteriser  les  personnes  et  les  animaux.  Cela  a  ete  realise  en  utilisant  des  enquetes  nationales  et  une  etude 
des  capteurs  de  detection  de  personnel  remontant  au  conflit  en  Asie  du  Sud-Est  entre  1956  et  1975. 
Les  capteurs  etudies  par  le  groupe  de  travail  ont  ete  les  microphones  audio  et  ultrasonores,  les  geophones, 
les  capteurs  passifs  a  infrarouge,  les  magnetometres,  les  radars  et  sonars,  les  profileurs  a  matrice 
pyroelectrique  et  les  cameras  basse  et  haute  resolution.  Dans  la  seconde  phase,  des  approches  de 
phenomenologie  des  capteurs  et  de  traitement  du  signal  ont  ete  examinees  et  developpees  au  niveau  de 
chaque  capteur.  Un  exercice  international  de  combat  a  simple  action  visant  a  recueillir  des  donnees  a  eu  lieu 
en  mars  2012  a  la  frontiere  sud-ouest  des  Etats-Unis  ;  il  a  reuni  beaucoup  d’entites  des  pays  participants. 
Ont  notamment  ete  presents  le  SASNet  du  Canada  et  le  systeme  Pearls  of  Wisdom  d’lsrael,  opere  en  lien 
avec  le  personnel  de  l’ARL.  Les  principaux  objectifs  de  l’essai  etaient  de  donner  l’occasion  de  collecter  des 
donnees  phenomenologiques  et  de  foumir  des  donnees  pour  l’essai  d’ algorithmes. 

Le  groupe  de  travail  SET-158  s’ est  reuni  en  de  nombreuses  occasions,  a  commencer  par  la  reunion 
inaugurale  qui  s’ est  tenue  au  sein  de  la  RTA  a  Paris,  en  avril  2010.  Deux  autres  reunions  ont  eu  lieu  apres  les 
reunions  de  la  commission  SET  en  Grece,  en  octobre  2010,  et  a  Quebec,  en  mai  2012.  Le  SET-158  a 
organise  la  quatrieme  reunion  technique  en  association  avec  le  groupe  de  travail  SET-153  et  le  groupe  de 
travail  SET-142  a  Cardiff,  le  1 1  octobre  2011.  Trois  reunions  supplementaires  ont  eu  lieu  en  relation  avec  les 
essais  de  recueil  de  donnees  sur  le  terrain  et  les  seminaires. 
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La  commission  a  organise  deux  seminaires  portant  sur  la  detection  de  personnel  aux  Etats-Unis,  en  lien  avec 
le  laboratoire  de  recherche  de  l’armee  de  terre  des  Etats-Unis.  Ces  activites  ont  attire  bon  nombre  de 
participants  et  de  nombreux  travaux  sur  la  detection  de  personnel  ont  ete  presentes.  Plus  de  deux  douzaines 
d’ articles  approuves  par  des  pairs  ont  ete  pub  lies  a  Tissue  des  activites  de  ce  groupe. 
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DISPOSABLE  MULTI-SENSOR  UNATTENDED  GROUND 
SENSOR  SYSTEMS  FOR  DETECTING  PERSONNEL 

Dr.  James  Sabatier1  and  Dr.  Thyagaraju  Damaria2 

1.0  INTRODUCTION 

The  effort  conducted  by  the  Task  Group  (SET-158  RTG)  on  Disposable  Multi-Sensor  Unattended  Ground 
Sensors  Systems  for  Detecting  Personnel  identified  and  accessed  technologies  to  detect  personnel  that  improved 
security  in  wide  area  situational  awareness  in  open  terrain  operations.  Existing  research  programs  on  sensor 
systems  for  personnel  detection,  as  well  as  novel  sub-systems  components  for  personnel  detection,  were  of 
primary  interest.  Sub-system  components  include  sensing  elements,  signal  processing  and  fusion  algorithms. 
Phenomenology  aspects  of  the  sensors,  the  environment  and  the  physics  of  human  motion  detection  were  the 
primary  drivers  of  the  research. 

Large  quantities  of  inexpensive,  unattended  ground  sensors  to  detect  personnel  in  hostile  environments  are 
of  broad  interest.  There  is  an  ongoing  need  to  research,  develop  and  demonstrate  low-cost  multi-sensor, 
non-imaging  sensor  nodes  which  can  be  disbursed  on  the  battlefield  to  classify,  localize  and  track  targets. 
Requirements  for  large  numbers  of  personnel  detection  sensors  demand  emphasis  on  new  inexpensive  classes  of 
sensing  elements,  while  maintaining  high  system  performance  levels  using  robust  signal  processing  and  fusion 
algorithms,  all  the  while  being  power  efficient. 

2.0  PROGRAM  OF  WORK 

The  SET-158  Task  Group  participating  Nations  were  Canada,  Denmark,  Finland,  France,  Netherlands,  Tunisia, 
Turkey  and  United  States.  The  Task  Group  used  three  objectives  to  accomplish  its  work.  The  first  objective  was 
to  review  current  and  historical  low-cost  sensors  available  for  personnel  detection  that  meet  the  above  rationale. 
This  was  accomplished  using  national  surveys  provided  by  participants  and  a  review  of  personnel  detection 
sensors  used  in  the  conflict  in  Southeast  Asia.  Briefly,  modem  sensors  for  personnel  detection  date  back  to  the 
war  in  Southeast  Asia  between  the  years  of  1956  -  1975.  One  principal  goal  was  the  interdiction  of  materiel 
moving  on  the  Ho  Chi  Minh  Trail.  Sensors  that  were  deployed  during  this  conflict  included  seismic  geophones, 
microphones,  electric  and  magnetic  sensors  and  radar.  The  target  signatures  of  interest  included  human  voice 
and  footsteps  and  the  sounds  and  vibrations  from  vehicles,  electric  and  magnetic  signatures  and  active  radar 
detection.  The  signals  received  are  fused  at  the  sensor  level.  For  example,  Navy-trained  sonar  operators  listened 
for  voices  on  microphones  and  attempted  to  correlate  the  presence  of  voice  signals  with  seismic  sensors  that 
detected  the  presence  of  footsteps.  Multi-node  systems  consisted  of  strings  of  geophones  to  obtain  target 
direction  and  speed  along  the  trail.  Additional  strings  of  geophones  were  used  to  confirm  the  target  presence  at 
distant  locations.  These  sensor  systems  included  long-  and  short-haul  radios  so  that  the  data  could  be  poled  from 
overhead  aircraft.  While  high  detection  rates  resulted,  the  systems  were  plagued  by  very  high  false  alarm  rates. 
These  efforts  were  supported  by  acoustic  and  seismic  working  groups  led  by  the  Naval  Air  Development  Center 
in  Warminster,  PA,  and  Sandia  National  Laboratory  in  Albuquerque,  NM.  The  interested  reader  is  referred  to 
several  readily  available  reports  titled  Project  CHECO  Report,  IGLOO  WHITE. 


1  Dr.  Sabatier  was  an  U.S.  Army  Research  Laboratory,  Intergovernmental  Personnel  Act  (IP A)  employee  at  the  time  this  SET  Panel 
research  was  conducted. 

Dr.  Thyagaraju  Damaria,  RDRL-SES-A,  U.S.  Army  Research  Laboratory,  Adelphi,  MD  20783,  USA. 
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Second,  a  review  of  the  physics  of  human  motion  research  that  impacted  personnel  detection  was  accomplished. 
This  included  the  historical  detection  approaches  used  in  sensing  mechanisms  such  as  human  voice,  seismic 
footsteps,  and  the  thermal  and  magnetic  phenomenon  of  walking  humans.  This  is  presented  in  Section  5  of  this 
report. 

Third,  relatively  new  sensing  modality  based  on  sensing  micro-Doppler  motion  of  human  body  appendages 
such  as  arm,  leg  and  torso  motion  was  exploited  (see  Section  4).  By  far  the  largest  modeling  effort  was  the 
development  of  the  micro-Doppler  velocity  signatures  from  human  limb  motion.  Considerable  parallel 
modeling  in  the  radar  community  is  concurrently  being  developed,  but  this  group  focused  on  the  ultrasonic 
Doppler  effects  of  limb  motion.  Ultrasound  provides  a  simple  and  low-cost  alternative  to  radar  for  range 
detection  of  a  few  tens  of  meters. 

Doppler  models  of  this  human  appendage  motion  were  developed  as  well  as  new  signal  processing  algorithms 
that  captured  the  physics  of  seismic  footsteps,  voice  when  present  and  the  micro-Doppler  information  from 
walking  people.  Logical  fusion  approaches  to  data  from  multiple  sensors  were  also  developed.  This  is  presented 
in  Section  6. 


3.0  SET-158  PROGRAMMATIC  EFFORTS 

To  further  advance  these  objectives,  the  Task  Group  took  advantage  of  workshops  on  personnel  detection  that 
were  jointly  organized  and  conducted  by  the  US  Army  Research  Laboratory  (ARL)  and  the  University  of 
Mississippi  (UM)  in  May  15-17,  2012.  ARL  and  UM  had  organized  two  previous  human  and  light  vehicle 
detection  workshops  prior  to  the  onset  of  SET- 158.  The  workshop  focus  was  human  detection  using  acoustic, 
seismic,  and  magnetic  and  electric  field  sensors  and  other  novel  sensing  mechanisms.  The  workshop’s  target 
audience  is  university,  industry  and  government  scientists  and  engineers  with  expertise  in  sensor  phenomenology, 
signal  processing,  detection  and  network  information  processing. 

A  NATO  RTO  Specialists’  Meeting  titled  “Autonomous  Sensing  and  Multi-Sensor  Integration  for  ISR 
Applications”  was  held  in  Cardiff,  UK,  24-25  October  2011.  The  NATO  technical  representatives  from 
SET-142  TG  on  “Acoustics  and  Autonomous  Sensing  for  ISR  Applications”,  SET-153  TG  on  “Multi-Sensor 
Integration  for  Urban  Operations”,  and  the  SET-158  TG  on  “Disposable  Multi-Sensor  Unattended  Ground 
Sensors  Systems  for  Detecting  Personnel”  developed  this  meeting  to  address  these  critical  technology  areas  in 
part  for  personnel  detection.  This  meeting  served  to  bring  together  experts  in  Doppler  human  motion  detection, 
signal  processing  and  fusion  for  personnel  detection. 

SET-158  conducted  task  force  meetings  for  planning  and  progress  reporting.  These  gatherings  led  the  Task 
Group  to  conduct  two  personnel  field  trials  on  the  SW  US  border  that  were  attended  by  national  representatives 
from  Canada,  Finland  and  the  US.  Several  additional  groups  participated  in  the  field  trial  including  University 
of  Memphis,  US  Army  Night  Vision  Laboratory,  Ft.  Belvoir,  VA  and  SPAWAR,  San  Diego,  CA.  ARL  and  UM, 
in  conjunction  with  University  of  Memphis  conducted  an  earlier  field  trial  on  the  SW  border  and  this  second  field 
trial  was  a  natural  follow-on  to  the  SET  Panel  activity. 


4.0  ULTRASONIC  MICRO-DOPPLER  MODELING 

Walking  people  create  unique  acoustic  Doppler  signatures  that  can  be  applied  to  human  motion  analyses. 
The  individual  body  segments  (foot,  leg,  arm,  torso,  etc.)  have  different  acoustic  cross-sections  and  velocities 
that  form  these  unique  human  Doppler  signatures.  Continuous-wave  ultrasound  is  used  to  measure  the  Doppler 
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shifted  signal  reflections  from  the  body  segments  as  the  person  moves.  The  reflected  ultrasonic  signal  from  the 
human  body  is  Frequency  Modulated  (FM)  by  the  motion  of  the  human  body  segments.  The  acoustic  cross- 
section  of  the  body  segment  influences  the  amplitude  of  the  received  Doppler  signal.  These  human  motion 
Doppler  signatures  have  been  extensively  studied  using  ultrasonic  sonar  systems. 

The  principle  of  ultrasonic  Doppler  sonar  relies  on  the  detection  of  a  Doppler  shift  in  the  frequency  of 
acoustic  waves  scattered  by  a  moving  target,  from  which  a  time -resolved  measurement  of  the  target  velocity  is 
obtained.  The  reflected  FM  ultrasound  signal  is  received  by  the  transducer  that  is  co-located  with  the  transmitter 
and  can  be  mathematically  expressed  as: 

git)  =  Acos(2nfct+  (pit))  (1) 

where  A  =  amplitude  of  the  received  signal,  fc  =  carrier  frequency,  and  (p  (t)  =  instantaneous  phase.  It  can  be 
written  as: 

0(0  =  y/^vCOdr  +  00 

where  v(t)  is  the  velocity  of  a  moving  object  and  can  be  written  in  terms  of  the  Doppler  frequency  as  in  the 
following: 

fait)  =  f  v(t)  (2) 

where  c  is  the  speed  of  sound. 

Generation  of  lit)  and  Q(t)  Samples 

In  order  to  demodulate  the  received  FM  signal,  the  FM  signal  is  converted  into  baseband  /(t)  and  Q  (  t)  by 
multiplying  the  FM  signal  with  both  the  carrier  signal  and  shifted  version  of  the  carrier  signal.  Mathematically, 
it  can  be  expressed  as: 

git)  x  gc  =  i[cos(47r/ct  +  (p)  +  cos(0)]  (3) 

where  gcit)  —  Bcosi2nfct)  and  is  synthetically  generated,  Q  (t)  samples  can  be  obtained  by  using  the 
following  expression: 

git)  x  gco  =  i[sin(47r/ct  +  0)  +  sin(0)]  (4) 

where  gcoit)  —  Bcos(f2nfct  —  n / 2)  ~  Bsini2nfct).  After  performing  low-pass  filtering  on  Eqs.  (3)  and 
(4),  the  baseband  /(t)  and  Q  (t)  are: 

lit)  =  cos  (0)  (5) 

Qit)  =  sini<p)  (6) 

The  Doppler  baseband  /(£)  and  Q  (t)  components  are  decimated,  to  reduce  the  over  sampling,  before  being 
added  to  form  a  complex  Doppler  signal  as  in  the  following: 

$it)  ~  I  dec  O')  +  IQdecif)  (2) 

where  7d ec  and  Qdec  are  the  decimated  /  and  Q  samples  and  i  is  the  imaginary  unit. 
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Velocity  Calculation  at  Base  Band 

The  velocity  of  human  body  segments  can  be  computed  using  a  well-known  spectrogram,  essentially  the  square 
of  the  Short-Time  Fourier  Transform  (STFT),  which  can  be  expressed  as: 

STFT(t,f )=  fS(t  +  t)w(t)  exp(— /27t/t)  dz  (8) 

where  S(t)  is  the  complex  down-converted  signal  as  given  in  Eq.  (7),  w(t)  is  a  sliding  window  function 
(e.g.,  a  Flamming  window),  t  is  time  and  /  is  frequency.  STFT  can  be  represented  as  a  time -frequency  plot. 
The  horizontal  axis  is  time,  the  vertical  axis  is  frequency,  and  the  magnitude  of  the  STFT  output  at  each  point  is 
represented  by  the  hue  of  the  point’s  color. 

4.1  Human  Motion  Sonar 

To  measure  the  human  motion  Doppler  signals,  the  ultrasonic  sonar  uses  commercially  available,  low-cost 
ultrasonic  ceramic  transducers.  The  sonar  was  assembled  from  two  identical  ultrasonic  transducers: 
one  transducer  acts  as  a  transmitter  emitting  an  ultrasonic  wave  while  the  other  acts  as  a  receiver,  sensing  the 
echoes. 

Measurements  of  the  sound  reflected  from  walking  people  using  the  sonar  were  conducted  on  a  straight  line 
track  marked  on  the  floor  in  a  hallway  of  a  modem  university  building.  The  sonar  was  placed  on  a  tripod  at  a 

1 .2  m  height  and  located  at  one  end  of  the  track.  In  the  test  configuration,  the  beam  patterns  of  the  ultrasonic 
transducers  were  oriented  along  the  walking  track  and  towards  the  walker.  The  person  started  walking  at  1  m 
from  the  sonar  location  and  walked  away  for  about  10  seconds  before  stopping  and  turning  around  to  walk  back 
towards  the  sonar.  The  walking  away  spectrogram,  or  the  sonar  signal,  is  presented  in  Figure  1  for  the  Doppler 
signature.  This  sonogram  is  the  composite  Doppler  motion  of  sound  scattering  from  the  whole  of  the  body 
segments  as  the  person  walks.  The  horizontal  line  at  zero  FIz  in  the  Figure  1  is  the  sum  of  direct  sound  coupling 
between  the  transmitter  and  the  receiver  through  the  air  and  the  common  enclosure  and  the  reflected  signals 
from  stationary  or  zero  velocity,  reflecting  objects.  The  strongest  amplitude  reflection  from  a  walking  person 
corresponds  to  the  fluctuating  line  near  the  frequency  of  -300  Hz  (marked  as  No.  1  in  Figure  1)  for  walking 
away  (negative  Doppler  shift)  from  the  sonar.  The  Doppler  shift,  -300  Hz,  is  proportional  to  the  person’s  speed 
V.  The  speed  of  the  walker  can  be  calculated  from  the  Doppler  shift  equation  F  =  2  fft/  A,,  where  V  is  the  speed, 
fd  is  the  Doppler  frequency  shift  and  A  is  the  acoustic  wavelength.  For  sound  speed  in  air  at  room  temperature, 
this  results  in  a  calculated  speed  of -1.3  m/s. 
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Figure  1 :  The  Composite  Doppler  Frequency  Response  of  a  Person 
Walking:  Torso,  Legs  and  Arms  Marked  as  Nos.  1,  2,  and  3. 
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The  envelopes  of  curves  marked  as  No.  2  in  Figure  1  correspond  to  the  motions  of  the  legs,  which  have  smaller 
acoustic  cross-sections  than  the  torso  and  therefore  have  lower  amplitudes.  The  leg’s  maximum  Doppler  shift, 
largest  velocity  (5.2  m/s),  is  from  the  foot  and  is  three  times  larger  than  the  torso  velocity.  The  motion  of  the 
arms  back  towards  the  sonar  as  the  person  walks  away  from  the  sonar  is  marked  as  the  envelope  No.  3  in  the 
figure  and  has  a  Doppler  shift  (hand  motion)  of  250  Hz  or  1  m/s.  The  motion  of  other  body  segments  results  in 
Doppler  returns  that  are  proportional  to  those  acoustic  cross-sections  and  velocities.  The  sonar  gram  shown  in 
Figure  1  is  a  whole  composite  Doppler  sonogram  of  all  of  the  moving  body  segments  from  which  the  ultrasound 
reflects  or  scatters.  To  further  understand  the  Doppler  contributions  of  the  individual  segments, 
a  Fresnel-Kirchoff  diffraction  theory  for  the  analysis  of  body  segment  motion  was  developed  and  is  presented 
next. 

4.2  Ultrasonic  Sonar  Model 

The  human  body  is  represented  as  a  segmented  link  system  following  Winter’s  model  [1],  The  Boulic-Thalmann 
model  [2]  is  used  to  predict  joint  angle  time  histories  and  the  temporal  displacements  of  the  body  center  of  mass. 
The  velocity  at  the  proximal  and  distal  end  of  each  key  body  segment  as  a  function  of  time  is  determined  from 
the  Boulic-Thalmann  (BT)  joint  rotations  and  body  translations  using  rigid  body  physics  for  the  angular  motion 
of  the  segment.  Scattering  is  assumed  to  occur  from  seven  types  of  body  segments:  the  foot,  lower  leg,  thigh, 
trunk,  head,  neck,  upper  arm  and  lower  arm-hand.  For  the  purposes  of  scattering,  the  body  segments  are  modeled 
as  ellipsoids.  The  dimensions  of  these  ellipsoids  are  estimated  from  the  segment  length,  mass  and  density. 
At-rest  acoustic  scattering  cross-sections  for  the  segments  are  determined  using  Fresnel-Kirchhoff  diffraction 
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theory.  Velocity  and  time  dependent  spectral  acoustic  cross  sections  are  obtained  by  exploiting  the  fact  that  the 
velocity  at  points  on  a  rotating-translating  rigid  body  must  continuously  vary  across  its  length.  Doppler  sonar 
grams  are  computed  by  mapping  the  time  dependent  spectral  acoustic  cross-sections  of  the  various  body 
segments  onto  time -velocity  space.  This  mapping  is  implemented  in  a  way  that  mimics  the  STFT  used  in  the 
processing  of  Doppler  sonar  data. 

A  walk  cycle  consists  of  two  steps.  It  begins  with  left  foot  Heel  Strike  (HS),  continues  through  right  foot  HS, 
left  foot  Toe  Off  (TO)  and  ends  with  the  second  HS  of  the  left  foot.  The  stride  period  is  the  time  required  to  take 
two  steps.  It  can  be  related  to  the  average  velocity  of  walking  and  the  body  height  H.  In  the  BT  model,  the  body 
is  represented  as  a  segmented  link  system  in  a  spatial  coordinate  system.  The  center  of  the  coordinate  system  is 
referred  to  as  the  spine  center.  The  lower  body  is  free  to  rotate  in  three  directions  (roll,  pitch  and  yaw)  about  the 
spine  center.  The  torso  is  free  to  rotate  about  the  spine  center  independently  of  the  lower  body.  Flexion  occurs  at 
the  hip,  knee  and  ankle  for  the  right  and  left  legs.  Flexion  also  occurs  at  the  shoulder  and  elbow  for  the  right  and 
left  arms.  Left-  and  right-side  leg  and  arm  motions  are  symmetrical.  Additionally,  the  body  translates  about  spine 
center.  In  total  there  are  twelve  degrees  of  freedom. 

The  model  assumes  a  measurement  geometry  in  which  an  individual  is  walking  directly  toward  the  sonar  - 
as  described  in  the  hallway  measurements.  This  geometry  maximizes  the  Doppler  sonar’s  response  to  sagittal 
plane  swinging  motions  of  the  legs  and  arms  as  well  as  sagittal  plane  motions  of  the  trunk.  When  the  body  is  in 
motion,  the  motion  induced  scattering  effects  include  the  assumption  that  the  segment  moves  like  a  rigid  body 
and  the  linear  velocities  of  scattering  centers  along  a  segment  length  can  be  computed  for  simple  rational 
dynamics  and  the  time  derivative  of  the  BT  joint  angle  time  function.  The  real  value  of  this  approach  is  that  it 
allows  for  the  signal  received  from  each  segment  to  be  modeled. 

Figure  2(a)  shows  the  contribution  to  a  Doppler  sonar  gram  from  the  right  foot  for  an  individual  walking  toward 
the  sonar  at  a  speed  of  1.3  m/s.  Heel  strike  of  the  left  foot  occurs  at  about  1.1  s.  At  this  time  both  feet  are  in 
contact  with  the  ground  and  the  velocity  is  zero.  Following  left  HS,  the  right  foot  swings  forward  producing  the 
wide  range  of  velocities  seen  between  1.25  and  1.5  s.  During  this  time  interval  there  are  substantial  differences 
between  the  velocity  of  the  ankle  and  the  toe.  These  differences  produce  the  velocity  spreads  at  individual  times 
in  the  gram  in  accordance  with  rigid  body  rotation.  Also  shown  in  Figure  2(a)  is  the  contribution  of  the  torso. 
The  torso  contribution  to  the  gram  occurs  in  a  very  narrow  velocity  band  due  to  the  small  differences  between 
the  velocity  of  the  spine  center  and  base  of  the  neck  (proximal  and  distal  segments  for  torso).  This  velocity 
oscillation  is  caused  by  accelerations  of  the  body  center  of  mass  resulting  from  two  heel  strikes  and  toe  offs  and 
is  twice  the  gait  frequency. 
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Figure  2(b)  shows  the  additional  contributions  to  the  Doppler  sonar  gram  from  the  right  lower  leg  and  thigh. 
The  contribution  to  the  sonogram  from  the  thigh  occurs  at  low  velocities  similar  to  those  of  the  torso  but  with  a 
broader  overall  spread  in  velocity  and  weaker  energy  levels  due  to  the  smaller  relative  size  of  the  thigh  in 
comparison  to  the  trunk.  The  return  from  the  lower  leg  fills  in  the  region  between  the  return  from  the  foot  and 
thigh.  Its  amplitude  is  stronger  than  that  of  the  foot  and  is  most  pronounced  when  the  lower  leg  is  perpendicular 
to  the  ground  and  normal  to  the  transducer.  At  these  times  the  lower  leg  presents  maximum  acoustic  cross- 
section  to  the  sonar.  The  torso  by  way  of  contrast  always  presents  maximum  acoustic  cross-section  to  the  sonar 
provided  that  the  individual  is  walking  directly  toward  or  away  from  the  sonar. 


4.3  Selected  Sonar  Publications 

M.  Bradley  and  J.M.  Sabatier,  “Distinguishing  Between  Human  and  Equine  Motion  Using  Micro  Doppler 
Sonar”,  Journal  Acoustical  Society  of  America  Express  Letters  EL-1 1-1567,  May  2012. 

M.  Bradley  and  J.M.  Sabatier,  “Acoustically-Observable  Properties  of  Adult  Gait”,  Journal  Acoustical  Society 
of  America  Express  Letters,  EL-1 1-1536,  2011. 

M.  Bradley  and  J.M.  Sabatier,  “Aerial  Ultrasonic  Micro  Doppler  Sonar  Detection  Range  in  Outdoor 
Environments”,  Journal  Acoustical  Society  of  America  Express  Letters,  EL-1 1-1512,  2011. 

M.  Bradley  and  J.M.  Sabatier,  “Applications  of  Fresnel-Kirchhoff  Diffraction  Theory  in  the  Analysis  of 
Human-Motion  Doppler  Sonar  Grams,”  Journal  Acoustical  Society  of  America  Express  Letters,  Vol.  128, 
pp.  1-10,  November  2010. 

A.  Mehmood,  J.M.  Sabatier,  M.  Bradley  and  A.  Ekimov,  “Extraction  of  Walking  Human’s  Body  Segments 
Using  Ultrasonic  Doppler”,  Acoustical  Society  of  America  Journal,  Vol.  128,  p.  316,  2010. 

S.G.  Benka,  “Doppler  SONAR  in  Air  for  Border  Security”,  Editor-in-Chief,  Physics  Today  June  2012, 
http://scitation.aip.Org/content/aip/magazine/physicstoday/news/l  0. 1 063/PT.4.040 1 . 
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5.0  FIELD  TRIAL:  DATA  COLLECTION  AT  A  US  SOUTHWEST  BORDER 

In  March  2012  several  Nations  and  universities  participated  in  data  collection  at  a  US  southwest  border  in 
order  to  develop  robust,  model-based  algorithms  for  personnel  detection.  The  selection  of  the  site  for  data 
collection  is  done  to  get  real  data  in  an  open  environment  where  the  algorithms  might  be  used  to  solve  the 
problems  faced  by  the  US  Department  of  Homeland  Security.  The  site  is  selected  because  the  border  security 
personnel  are  faced  with  actual  encounters  with  people  crossing  the  US  border  illegally.  Some  other 
considerations  involved  in  site  selection  are: 

a)  Different  terrains  (for  example,  hard  tracks  and  soft  sand  river  bed  used  by  the  trespassers); 

b)  Environmental  diversity;  and 

c)  Wildlife  (which  causes  false  alarms  on  fielded  sensors),  etc. 

The  participants  of  the  data  collection  are  listed  below: 

1)  US  Army  Research  Laboratory  (ART),  Adelphi,  MD  20783,  USA. 

2)  Night  Vision,  Ft.  Belvoir,  VA,  USA. 

3)  SPA  WAR,  San  Diego,  CA,  USA. 

4)  University  of  Mississippi,  Oxford,  MS,  USA. 

5)  University  of  Memphis,  Memphis,  TN,  USA. 

6)  Canadian  Defence  Organization. 

7)  Israeli  Team. 

8)  Finnish  Defence  Organization. 

5.1  ARL  Data  Collection  System 

ARL  has  brought  three  sensor  systems  which  are  used  to  collect  the  data  for  the  choreographed  scenarios. 
It  also  brought  another  three  sensor  systems  to  collect  data  of  animals  in  their  natural  habitat;  the  data  is 
collected  day  and  night.  The  sensor  systems  are: 

1)  Wavebook  data  collection  system  is  primarily  used  for  data  collection  for  choreographed  scenarios 
where  people  and  animals  walked  along  the  trails  in  an  orderly  fashion.  It  has  8  channels  for  data 
acquisition.  The  sampling  rate,  and  the  aliasing  filters  can  be  pre-programmed  as  desired.  The  following 
sensors  are  used  on  the  Wavebook: 

•  Acoustic; 

•  Seismic; 

•  Passive  Infrared  (P1R);  and 

•  Ultrasonic  sensor  suite. 

2)  Automatic  data  collection  unit  is  used  at  remote  sites  to  collect  the  data  round  the  clock  for  animals  in 
their  natural  habitat.  The  system  is  capable  of  collecting  data  on  8  channels  at  4  k  samples  per  second. 
The  sensor  suite  consists  of: 

•  Acoustic; 
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•  Seismic; 

•  PIR;  and 

•  Ultrasonic  sensor  suite. 

The  sensors  -  acoustic,  seismic,  PIR  and  ultrasonic  -  used  for  data  collection  are  the  same  as  the  ones  used 
during  the  2009  data  collection.  The  Wavebook  data  collection  system  with  the  sensor  suite  deployed  is  shown 
in  Figure  3  and  Figure  4  shows  one  of  the  scenarios  being  enacted  during  the  data  collection.  Figure  4  clearly 
shows  several  people  and  animals  walking  the  trail. 


(a)  (b) 

Figure  3:  (a)  ARL  Sensor  Deployment;  (b)  New  Profiling  Sensor. 


Figure  4:  People  and  Animals  Walking  on  the  Trail. 
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5.2  Night  Vision  Data  Collection  System 

Night  Vision,  Fort  Belvoir,  VA,  has  brought  a  high-resolution  camera  to  the  field  to  collect  data.  The  camera 
system  is  shown  in  Figure  5.  The  system  included  a  fisheye  to  see  the  targets  coming  from  all  around. 


Figure  5:  Night  Vision  Camera  System. 


5.3  SPAWAR  Data  Collection  System 

SPAWAR  has  brought  two  magnetic  sensors  and  deployed  them  along  the  trail.  The  sensor  is  sensitive  enough 
to  detect  people  passing  by  carrying  some  ferrous  material  (for  example,  keys).  The  sensors  are  shown  in 
Figure  6. 


Figure  6:  SPAWAR’s  Magnetic  Sensors. 
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5.4  University  of  Mississippi  Data  Collection  System 

University  of  Mississippi  has  deployed  a  similar  system  to  that  of  ARLs  with  acoustic,  seismic  and  ultrasonic 
sensors.  The  deployed  system  is  shown  in  Figure  7. 


Figure  7:  University  of  Mississippi  Sensors. 


5.5  University  of  Memphis  Data  Collection  System 

The  University  of  Memphis  had  been  working  on  developing  a  new  sensor  system  to  replace  high  power 
consuming,  high  bandwidth  requiring  imaging  sensors  such  as  a  camera.  They  have  developed  a  pyro-electric 
array  profiling  sensor  with  fewer  pixels  to  capture  the  essence  of  an  image.  The  deployed  sensor  is  shown 
in  Figure  8. 


Figure  8:  Profiling  Sensor  by  University  of  Memphis. 
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5.6  Defence  Research  and  Development  Canada  SASNet 

The  Canadian  Defence  Organization  has  developed  a  low-cost  network  called  “Self-healing  Autonomous 
Sensor  Networks  (SASNet)  for  detection  and  tracking  of  targets.  Each  sensor  node  in  the  SASNet  consists  of 
acoustic,  seismic  and  magnetic  sensors.  Figure  9  shows  some  elements  of  SASNet. 


Figure  9:  SASNet. 


5.7  Israeli  Team 

The  Israeli  team  has  deployed  their  system  called  “Pearls  of  Wisdom”  consisting  of  acoustic,  seismic,  magnetic 
and  imaging  sensors.  Their  system  is  shown  in  Figure  10. 


Figure  10:  Pearls  of  Wisdom. 
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6.0  SIGNAL  PROCESSING 

In  this  section,  we  present  some  of  the  advances  made  in  seismic  and  ultrasonic  signal  processing  for  personnel 
detection.  Most  of  the  advances  in  signal  processing  concentrated  on  acoustic  (ultrasound)  and  seismic  sensors 
as  they  offer  high  fidelity  to  distinguish  people  from  animals. 

6.1  Seismic  Signal  Processing 

The  main  purpose  of  a  seismic  sensor  is  to  detect  footfalls  of  humans  walking  within  the  receptive  field  of  the 
sensor.  There  is  a  considerable  amount  of  literature  [3],  [5]  in  footstep  detection.  Traditionally,  researchers  have 
focused  on  estimating  the  cadence.  However,  if  multiple  people  are  in  the  vicinity  of  the  sensor  and  walking, 
it  is  difficult  to  estimate  the  cadence  of  an  individual  person.  Moreover,  if  there  are  animals,  it  is  difficult  to 
differentiate  multiple  people  walking  and  animals  walking  by  observing  the  footfalls.  However,  multiple 
footfalls  superimposed  on  one  another,  resulting  in  a  frequency  of  ‘c’  Hz  (where  ‘c’  is  an  effective  cadence  of 
multiple  walkers).  So,  a  seismic  algorithm  can  look  for  harmonics  of  cadence  or  several  strong  frequency 
components  between  2  to  15  Hz  to  distinguish  single  and  multiple  walkers. 

The  seismic  algorithm  used  is  a  multivariate  Gaussian  classifier  [4]  with  the  feature  set  consisting  of  amplitudes 
of  the  frequency  bins  from  2  to  15  Hz.  Then  the  algorithm  is  used  to  estimate  the  posterior  probability  of  footsteps 
present. 

The  algorithm  only  determines  whether  there  are  footsteps  present.  In  order  to  detect  the  presence  of  humans, 
it  is  necessary  to  determine  whether  these  footsteps  belong  to  a  human  or  an  animal.  For  this,  we  invariably 
turn  to  acoustics.  If  there  is  voice,  it  can  be  detected  and  identified  as  a  human  voice  based  on  the  formants. 
In  order  to  distinguish  people  and  animals  when  no  voice  is  present,  we  analyze  the  sound  generated  by  the 
animals  walking.  When  a  single  hoof  of  a  horse  strikes  the  ground,  it  produces  a  sound  pattern  that  is 
distinct  from  that  of  a  human  foot.  Figure  1 1(a)  shows  the  signature  of  a  horse  walking  (for  a  period  of  6  s) 
before  and  after  noise  removal.  The  noise  removal  is  performed  using  empirical  mode  decomposition  of  the 
original  signal  into  various  component  signals.  From  Figure  1 1(b),  it  is  clear  that  there  are  three  peaks  uniformly 
distributed  in  each  time  interval  of  one  second.  This  indicates  the  cadence  of  the  horse  to  be  approximately  2.8  to 
3  Hz.  Since  the  cadence  of  a  person  is  around  1 .5  to  2  Hz,  one  can  infer  the  presence  of  animals. 
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Figure  11:  (a)  Hoof  Signature;  (b)  Hoof  Signature  After  Noise  Removal. 


When  a  person  walks,  the  heel  of  the  foot  strikes  first  and  then  the  toe  end  of  the  foot  strikes,  rubbing  against 
the  ground,  creating  a  unique  seismic  signature  compared  to  that  of  an  animal  (see  Figure  12)  Animals, 
in  general,  walk  on  their  hoof  or  “toe”  (the  horse  ankle  and  heel  or  fetlock  do  not  strike  the  ground),  which 
strikes  the  ground  producing  signatures  that  are  different  from  those  of  people.  Both  the  signatures  have 
different  frequency  responses  on  the  same  ground.  In  Section  6.2,  we  present  a  technique  that  uses  the 
differences  in  frequency  responses  to  distinguish  both  types  of  footfalls. 

Single  footstep 


Front  leg 


Back  leg 


Figure  12:  Seismic  Signature  of  a  Single  Footstep  of  a  Person. 
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6.2  Discrimination  of  Animal  and  Human  Seismic  Signatures 

For  a  single  walking  person,  detection  of  cadence  and  human  footstep  signature  is  relatively  easy.  However, 
when  animals  and  people  are  walking  in  the  vicinity  of  a  seismic  sensor,  the  detection  of  human  foot 
signature  is  not  as  straightforward.  If  there  are  multiple  sensors  collecting  the  same  signatures,  one  can  use  the 
Principal  Component  Analysis  (PCA)  or  Independent  Component  Analysis  [6]  (ICA)  to  separate  the  human 
footstep  signatures  from  the  animal  footsteps  depending  on  whether  the  noise  is  Gaussian  or  not.  Most  of 
the  Unattended  Ground  Sensor  (UGS)  systems  consists  of  only  one  sensor  per  modality,  that  is,  one  acoustic, 
one  seismic,  etc.,  so  it  is  not  possible  to  use  the  PCA  or  ICA  technique  for  signal  separation,  since  PCA 
or  ICA  requires  at  least  ‘n+ V  number  of  sensors  to  separate  'n'  sources.  In  acoustics,  several  researchers  [7] 
have  developed  techniques  for  single  channel  source  separation  where  they  attempted  to  separate  signals  from 
two  human  speakers  from  a  single  microphone.  In  almost  all  the  cases,  they  used  Short  Time  Fourier  Transform 
(STFT)  and  Non-negative  Matrix  Factorization  (NMF)  techniques. 

The  NMF  technique  was  first  introduced  by  Lee  and  Seung  [8]  and  was  adopted  by  others  to  minimize  the  cost 
function: 


\ 

2 


Z 


+^ZK, 


(9) 


where  X  is  the  STFT  with  variables  in  frequency  w  and  time  t,  H  and  W  are  the  basis  and  weight  matrices,  and  X 
controls  the  sparcity  of  the  weights,  that  is,  fewer  weights,  hence  fewer  basis  functions,  will  be  used.  The  fact 
that  the  elements  of  Xw  t ,  H  and  W  are  all  non-negative  gives  the  algorithm  the  name  non-negative  matrix 

factorization.  We  use  Discrete  Cosine  Transform  (DCT)  instead  of  STFT  to  avoid  the  problems  arising  due  to 
complex  signals.  Let: 

Xi=dct(xi(t))  (1Q) 

be  the  DCT  of  the  signal  xi  (t ) .  It  is  found  that  the  first  500  of  the  DCT  coefficients  are  sufficient  to  reconstruct 
the  time  domain  signal  with  negligible  distortion.  It  is  worth  noting  that  earlier  versions  of  JPEG  compression 
schemes  used  DCT.  So  Xt  denotes  the  first  500  DCT  coefficients.  Let  Bi  and  /F  denote  the  positive  and 

negative  DCT  coefficients  such  that  X ;  =  Bt  —  .  Let  the  matrix  X p  =  [Xi };  V i  be  the  set  of  DCT 

coefficients  for  all  the  training  data  corresponding  to  the  people.  Then,  the  matrix  \X  ]  can  be  written  as: 

[xp}  =  [x;-\-[x~p\ 


with  matrix  [Xp  ]  =  \Bi J  representing  the  positive  DCT  coefficients  and  matrix  \X  ]  =  {/?■ }  representing  the 


negative  DCT  coefficients  of  X  .  Similarly,  X a  represents  the  set  for  animals.  After  performing  the  NMF  on 
the  matrices,  we  get: 
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The  matrices  W  and  W  represent  the  weight  matrices  and  the  matrices  H  and  H  correspond  to  the  bases. 
Once  the  basis  matrices  are  available,  they  can  be  used  to  represent  the  DCT  coefficients  X,  of  a  test  signal  x(t)  as 
the  weighted  sum  of  their  components.  The  algorithm  to  estimate  the  weights  and  bases  (subset  of  H  and  H )  is 
given  below. 

Algorithm  1: 

•  Step  1 :  Normalize  the  test  signal  x(t)  after  removing  the  mean.  Compute  X,  =  dct  (x(t)).  Let  X,  =  B, 
where  B,  and  ft,  denote  the  positive  and  negative  DCT  coefficients. 

•  Step  2:  Estimate  the  weights  W  =  {wj ,  w2 ,  •  •  • ,  wr }  and  V  =  {vx ,  v2 ,  ■  •  • ,  vr }  such  that: 

iBt  -  WH|2+ |pt  -  VH|2;  such  that  0<©i,Vi<ub;  Vie  {1,2,-  •  -  ,r}  (11) 

is  minimum,  ub  is  typically  1.  One  may  use  any  constrained  non-linear  optimization  program  such  as  the 
“fmincon”  function  in  MATLAB  to  determine  the  weights: 

•  Step  3:  Non-zero  weights  co  and  v  give  the  bases  used  to  represent  X,. 

•  Step  4:  Reconstruct  the  signal  x(t )  by  taking  the  inverse  DCT  of  the  difference  (co  H-W  H). 

We  used  the  NMF  technique  to  separate  human  and  animal  signatures  from  a  single  seismic  channel  data. 
In  order  to  verify  the  technique,  we  took  two  seismic  signals,  one  from  a  person  and  another  from  a  horse 
walking,  and  mixed  them  as  shown  in  Figure  13.  Then  we  used  the  NMF  technique  on  the  mixture  to 
separate  the  signals.  The  results  of  the  NMF  technique  are  shown  in  Figure  14  and  Figure  15.  From  these 
figures  we  notice  the  reconstruction  (separation)  of  the  signals  is  good  except  in  the  places  where  there  is  noise. 

Footfall  Signature  of  a  person 


Figure  13:  Mixture  of  a  Signature  from  a  Person  and  a  Horse. 
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Reconstructed  Persons  Signature 


1.5  2 

sample  num. 


xIO 


Figure  14:  Extraction  of  Human  Signature. 


Reconstructed  Horse  Signature 


1.6  2 
sample  num. 


xIO 


Figure  15:  Extraction  of  Horse  Signature. 
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The  NMF  algorithm  is  further  used  on  several  sets  of  data  collected  on  a  single  person,  single  horse,  and  both  a 
horse  and  a  person  walking.  The  sum  of  the  weights  corresponding  to  the  bases  of  horse  Sh  and  a  person  Sp 
determine  whether  the  extracted  signature  belongs  to  a  horse  (animal)  or  a  person  depending  on  whether 
Sh  >  Sp.  The  values  of  Sh  and  Sp  are  plotted  in  16  as  ‘o’  and  **’,  respectively.  From  the  figure  we  find  that  the 
NMF  algorithm  provided  Sp  >  Sh  and  is  higher  than  the  threshold  shown  by  a  solid  line  at  0.7  for  the  data  with 
one  person  walking  majority  of  times  and  Sh  >  Sp  for  the  case  when  a  horse  was  walking.  When  both  a  person 
and  a  horse  walked,  both  Sh  and  Sp  are  above  the  threshold,  indicating  that  both  the  targets  are  present.  So  we 
can  detect  and  classify  the  footprint  signatures  of  people  and  animals  even  when  both  are  present  at  the  same 
time.  Traditional  classification  algorithms  classify  only  any  one  of  the  targets  present  but  not  both  simultaneously. 
They  fail  if  both  targets  are  present. 


Man  Walking 


Time  (sec) 


Figure  16:  Results  of  NMF  Algorithm  on  Signature  Data  of 
(a)  Man,  (b)  Horse,  and  (c)  Man  and  Horse  Walking. 


6.3  Ultrasonic  Sensor  Modeling  and  Ultrasonic  Signal  Processing 

In  this  section,  we  present  the  analysis  of  ultrasonic  sensor  data  to  characterize  and  discriminate  both  people 
and  animals.  This  is  an  active  sensor  that  radiates  a  40-kHz  ultrasonic  signal  and  captures  the  signals  that 
are  bounced  back  by  the  target  in  its  beam.  The  principle  of  operation  is  same  as  radar  [10].  The  micro-Doppler 
returns  due  to  the  swinging  of  the  arms,  legs,  and  torso  of  a  person  or  an  animal  are  analyzed.  We  take 
advantage  of  these  Doppler  returns  from  the  limbs  to  classify  the  targets.  Flowever,  in  order  to  understand  the 
type  of  Doppler  signatures  that  woidd  be  generated  by  the  swinging  of  arms,  legs,  and  torso  of  various  targets, 
it  is  necessary  to  model  these  parts  and  compute  the  Doppler  values. 
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Our  University  of  Mississippi  partners  Bradley  and  Sabatier  [9]  have  explained  the  observed  human-gait 
features  in  Doppler  sonar  grams  by  using  the  Boulic-Thalmann  (BT)  model  [2],  shown  in  Figure  17,  to  predict 
joint  angle  time  histories  and  the  temporal  displacements  of  the  body’s  center  of  mass.  In  the  BT  model, 
body  segments  are  represented  as  ellipsoids.  Temporally  dependent  velocities  at  the  proximal  and  distal  end  of 
key  body  segments  are  determined  from  the  BT  model,  as  shown  in  Figure  18.  Doppler  sonar  grams  are 
computed  by  mapping  velocity-time  dependent  spectral  acoustic  cross-  sections  for  the  body  segments  onto 
time-velocity  space,  mimicking  the  STFT  used  in  Doppler  sonar  processing.  Figure  19(a)  shows  the  estimated 
velocities  using  the  model  for  various  parts  of  the  body  for  a  6-foot-tall  person. 


Human  Segment  Model 


(a) 


Segmented  link  model  -  horse 


Figure  17:  (a)  Human  Model  as  a  Stick  Figure  and  (b)  Horse  Model. 


Figure  18:  Estimation  of  Velocities  of  Various  Parts  of  Body. 
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Figure  19:  Estimated  Velocity  (Doppler)  for  (a)  a  Person  Walking  and  (b)  a  Horse  Walking. 


The  Doppler  is  related  to  the  radial  velocity  vr  of  the  target  and  is  given  by  fd  = — ~fc,  where  c  is  the 

c 

propagation  velocity  of  sound  and  fc  is  the  radiated  carrier  frequency.  The  detailed  computation  of  velocities 
of  various  limbs  can  be  found  in  [9].  Similar,  models  are  developed  for  a  quadruped  [11],  such  as  a  horse. 
Figure  19(b)  shows  the  estimated  velocities  for  a  horse. 

Figure  20  shows  the  Doppler  signature  collected  for  a  person  walking  in  one  of  the  scenarios.  Each  sensor 
records  the  data  when  the  person  walks  by  the  sensor.  The  data  collection  is  done  when  a  person  (or  a  horse) 
walks  towards  the  sensor  at  close  range  (Figure  20(b))  and  when  the  person  (or  a  horse)  walks  away  from  the 
sensor  at  a  distance  and  to  one  side  of  the  sensor  (Figure  20(c)).  In  the  first  case,  the  signal  strength  is  high  and 
the  Doppler  returns  from  the  various  parts  are  clearly  visible,  while  in  the  latter  case  the  Doppler  returns  are 
weak  and  the  features  are  not  clearly  visible.  Hence,  we  developed  two  algorithms  to  classify  the  targets, 
namely: 

a)  When  the  Signal-to-Noise  Ratio  (SNR)  is  high  (>  6  dB);  and 

b)  When  the  SNR  is  low. 
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(a) 


(b) 


(c) 


Figure  20:  (a)  Doppler  Returns  for  a  Person  Walking,  (b)  Measured  Doppler  when  a  Person  Walks 
Towards  the  Sensor  and  (c)  Doppler  Returns  when  a  Person  Walks  Away  from  the  Sensor. 


Doppler  rotuns  from  a  person  walking 


Case  1:  High  Signal-to-Noise  Ratio  (SNR) 

In  this  case,  the  targets  walk  in  front  of  the  sensor  at  a  close  proximity,  and  the  atmospheric  (wind)  effects  on 
the  received  signal  are  minimal.  This  is  the  case  where  some  model  features  can  be  clearly  identified, 
and  the  classification  can  be  made  based  on  the  model.  An  example  of  high  SNR  is  shown  in  Figure  20(b). 
In  order  to  characterize  the  target  either  as  a  person  or  an  animal,  we  look  for: 

a)  Cadence; 

b)  Maximum  and  minimum  variation  in  the  Doppler  frequency  due  to  limbs;  and 

c)  The  sequential  nature  of  limb  movements. 

Figure  19  shows  the  enlarged  version  of  Figure  20(b).  It  shows  the  average  Doppler  of  the  torso  (average 
velocity  of  a  person  walking)  and  the  Doppler  due  to  arms  and  legs.  When  the  arms  and  legs  swing  forward/ 
backward,  we  get  a  Doppler  above/below  the  average  (the  sinusoidal  line  above/below  the  average  line). 
The  cadence  is  estimated  as  1  It,  where  t  is  the  time  between  two  peaks  of  a  sinusoidal  curve.  The  cadence  is 
estimated  to  be  1.8  Hz  for  the  person.  The  maximum  and  minimum  Doppler  frequency  of  limbs  with  respect 
to  the  average  is  found  to  be  ±800  Hz,  and  this  will  be  contrasted  with  the  values  for  an  animal.  The  vertical 
lines  on  Figure  21  are  drawn  to  show  the  lag  (sequential  nature  of  limb  movement)  between  the  top  and  lower 
sinusoidal  curves.  The  lag  is  ~  0.1  s. 
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Doppler  returns  from  a  person  walking 


4  4.5  5  5.5  6  6.5 

Time  (sec) 

Figure  21 :  Measured  Doppler  Output  for  a  Person. 


Figure  22  shows  the  ultrasonic  returns  from  a  horse  walking.  One  clear  distinction  between  the  Doppler 
signatures  for  a  person  and  a  horse  walking  is  that  the  signature  for  a  person  is  sinusoidal  in  nature. 
Horse  motion  is  much  more  complex,  as  there  are  many  more  moving  parts  for  a  horse  than  a  person. 
Another  distinct  feature  for  the  horse  is  that  the  Doppler  below  the  average  torso  Doppler  is  significantly 
less.  The  maximum  variation  of  Doppler  for  the  horse  (-1500  Hz)  is  higher  compared  to  that  for  a  person 
(-1100  Hz).  Figure  23  shows  the  Doppler  energy  plot  for  a  horse  walking.  The  peaks  in  Figure  23  show  the 
periodic  nature  of  a  horse  walking;  the  cadence  can  be  estimated  from  it.  The  cadence  of  the  horse  is  estimated 
to  be  around  -1.7  Hz.  This  cadence  value  is  significantly  low  for  a  horse  because  the  horse  is  made  to  walk 
slowly  on  purpose.  This  is  also  verified  using  the  seismic  data.  The  cadence  of  the  horse  is  close  to  the 
cadence  of  a  person  walking.  Hence,  cadence  alone  cannot  be  used  to  distinguish  a  person  from  a  horse  or 
any  other  quadruped.  From  Figure  23,  we  notice  that  each  peak  has  an  adjacent  smaller  peak  marked  by 
ellipses  in  the  figure.  This  double  peakedness  is  characteristic  of  a  quadruped  walking  and  is  also  seen  in  the 
model  shown  in  Figure  19(b).  The  time  difference  between  two  adjacent  peaks  is  -0.12  s.  The  algorithm  to 
distinguish  people  and  animals  using  the  features  described  earlier  is  given  in  the  flowchart  shown  in 
Figure  24. 
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Doppler  return*  from  a  horse  walking 


Tim*  (sec) 

Figure  22:  Measured  Doppler  for  a  Horse. 


Enorgy  In  500  -  1 100  Hz  band 
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Ultrasonic  Data 


Figure  24:  Classification  of  Ultrasonic  Data. 


Case  2:  Low  Signal-to-Noise  Ratio 

In  the  event  the  signal  returns  are  weak  for  any  reason  (such  as  prevailing  winds,  the  target  being  farther  than 
optimal  distance,  the  target  being  illuminated  by  the  ultrasonic  transducer  at  an  angle,  etc.),  the  features 
observed  in  the  case  of  high  SNR  may  not  be  present  as  seen  in  Figure  20(c).  For  low  SNR  data,  it  is  appropriate 
to  use  classical  signal  processing  techniques  to  classify  the  targets.  We  used  a  support  vector  machine  for 
classification,  as  shown  in  Figure  23. 

6.4  Selected  Publications 

1)  T.  Damarla,  M.  Bradley,  A.  Mehmood  and  J.M.  Sabatier,  “Classification  of  Animals  and  People  Ultrasonic 
Signatures”,  IEEE  Sensors  Journal,  Vol.  13,  No.  5,  pp.  1464-1472,  May  2013. 

2)  A.  Mehmood  and  T.  Damarla,  “Kernel  Non-Negative  Matrix  Factorization  for  Seismic  Signature 
Separation”,  Journal  of  Pattern  Recognition  Research  Vol.  8,  No.  1,  pp.  13-25,  June  2013. 

3)  S.  Bahrampour,  A.  Ray,  S.  Sarkar,  T.  Damarla  and  N.M.  Nasrabadi,  “Performance  Comparison  of  Feature 
Extraction  Algorithms  for  Target  Detection  and  Classification”,  Pattern  Recognition  Letters,  June  2013. 

4)  A.  Mehmood,  T.  Damarla  and  J.M.  Sabatier,  “Separation  of  Human  and  Animal  Seismic  Signatures 
Using  Non-Negative  Matrix  Factorization”,  Pattern  Recognition  Letters  Vol.  33,  Issue  16,  pp.  2085-2093, 
December  2012. 

5)  T.  Damarla  and  L.  Kaplan,  “A  Fusion  Architecture  for  Tracking  a  Group  of  People  using  a  Distributed 
Sensor  Network”,  Proc.  1S1F  Fusion  2013  Conference,  Istanbul,  Turkey,  July  2013. 

6)  T.  Damarla  and  A.  Mehmood,  “Detection  of  Targets  Using  Distributed  Multi-Modal  Sensors  with  Correlated 
Observations”,  Proc.  IEEE  Sensors  Conference,  Baltimore,  MD,  USA,  November  2013. 

7)  T.  Damarla,  R.  Frankel,  H.  Vu,  M.  Thielke,  A.  Mehmood,  J.  Sabatier  and  G.  Chatters,  “Personnel 
Detection  at  a  Border  Crossing  -  An  Exercise”,  ARL  Tech  Report  September  2013. 

7.0  SUMMARY 

SET-158  accomplished  three  main  tasks.  First,  this  includes  a  significant  multi-modal,  multi-national  data 
collection  exercise  was  conducted  at  the  southwest  border  of  the  US.  Second,  physics  based  models  of  the 
micro-Doppler  motion  of  human  and  animal  appendages  were  developed.  Thirdly,  new  signal  processing 
approaches  to  the  Doppler  data  were  developed  and  these  data  were  fused  with  other  sensing  modalities.  All  of 
the  research  accomplished  has  been  extensively  published  in  peer-reviewed  journals. 
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